Quantopian's community platform is shutting down. Please read this post for more information and download your code.
Back to Community
How can I use `fetch_csv` to load my data and make it available to the `history` function?

Basically I want to load some data into my algo using 'fetch_csv' and I also need the recent bars in 'handle_data'. This security symbol is outside of Quantopian's symbol database. I tried to supply the data as 'Security Information' or 'Signals', but neither works.

For example:

def initialize(context):  
    context.i = 0  
    context.window = 5  
    url = 'http://7u2o2u.com1.z0.glb.clouddn.com/@/csi300/csi300.csv'  
    fetch_csv(url, mask=False, symbol='CSI300')  
    # url = 'http://7u2o2u.com1.z0.glb.clouddn.com/@/csi300/csi300_symbol.csv'  
    # fetch_csv(url, mask=False)  
    # context.symbol = symbol('CSI300')  
    add_history(4, '1d', 'high')  
    add_history(4, '1d', 'low')  

def task1(high, low):  
    return False

def task2(high, low):  
    return False  

def handle_data(context, data):  
    # Skip first 4 days to get full windows  
    context.i += 1  
    if context.i < context.window:  
        return

    high_hist = history(4, '1d', 'high')  
    low_hist = history(4, '1d', 'low')  

At the last two lines, only AAPL history data is available.

The 'csi300.csv' looks like:
date,open,high,low,close,volume,price 2014-01-02 00:00:00+00:00,2323.433,2325.991,2310.653,2321.978,4519429100.0,2321.978

and 'csi300_symbol.csv' looks like:
date,open,high,low,close,volume,price,symbol 2014-01-02 00:00:00+00:00,2323.433,2325.991,2310.653,2321.978,4519429100.0,2321.978,CSI300

How should I do with this please?

4 responses

Hi Shel,

Sorry for the trouble - there are a two gotchas with fetcher and history. The attached backtest does what you want by using workarounds. I think it is easier to follow the work arounds in the code, but the two problems are:

  • fetching signal data only can result in an empty universe, which means handle_data is never called
  • history only accumulates data for trade bars from Q, so you have to build your own frame of data on fetcher fields

thanks,
fawce

edit: I forgot to mention that I made a new csv matching csi300 described above and shared it on dropbox for this test.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

-> history only accumulates data for trade bars from Q, so you have to build your own frame of data on fetcher fields

this is really unfortunate; hopefully this could be improved once 'total returns' (including dividends per instrument) is implemented.

Well it is a little tricky, got the idea.

Thank you @Fawce and @Chandelier, your replies are very very helpful.

This is contradicted by the current documentation which implies you can use the history function with fetched CSV data.

Best of all, your Fetcher data will play nicely with Quantopian's other data features:

Create a trailing window of data using history to make statistical models using your fetcher data.

What does the above mean?